Improving Tree-Structured Decoder Training for Code Generation via Mutual Learning

نویسندگان

چکیده

Code generation aims to automatically generate a piece of code given an input natural language utterance. Currently, among dominant models, it is treated as sequence-to-tree task, where decoder outputs sequence actions corresponding the pre-order traversal Abstract Syntax Tree. However, such only exploits based preceding actions, which are insufficient ensure correct action predictions. In this paper, we first throughly analyze context modeling difference between neural models with different traversals decodings (preorder vs breadth-first traversal), and then propose introduce mutual learning framework jointly train these models. Under framework, continuously enhance both two via distillation, involves synchronous executions one-to-one knowledge transfers at each training step. More specifically, alternately choose one model student other its teacher, require fit data prediction distributions teacher. By doing so, can fully absorb from thus could be improved simultaneously. Experimental results in-depth analysis on several benchmark datasets demonstrate effectiveness our approach. We release https://github.com/DeepLearnXMU/CGML.

برای دانلود باید عضویت طلایی داشته باشید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Neuroprosthetic Decoder Training as Imitation Learning

Neuroprosthetic brain-computer interfaces function via an algorithm which decodes neural activity of the user into movements of an end effector, such as a cursor or robotic arm. In practice, the decoder is often learned by updating its parameters while the user performs a task. When the user's intention is not directly observable, recent methods have demonstrated value in training the decoder a...

متن کامل

Natural Language to Structured Query Generation via Meta-Learning

In conventional supervised training, a model is trained to fit all the training examples. However, having a monolithic model may not always be the best strategy, as examples could vary widely. In this work, we explore a different learning protocol that treats each example as a unique pseudo-task, by reducing the original learning problem to a few-shot meta-learning scenario with the help of a d...

متن کامل

Learning Tree Structured Potential Games

Many real phenomena, including behaviors, involve strategic interactions that can be learned from data. We focus on learning tree structured potential games where equilibria are represented by local maxima of an underlying potential function. We cast the learning problem within a max margin setting and show that the problem is NP-hard even when the strategic interactions form a tree. We develop...

متن کامل

Scale Up Event Extraction Learning via Automatic Training Data Generation

The task of event extraction has long been investigated in a supervised learning paradigm, which is bound by the number and the quality of the training instances. Existing training data must be manually generated through a combination of expert domain knowledge and extensive human involvement. However, due to drastic efforts required in annotating text, the resultant datasets are usually small,...

متن کامل

Improving certification capability through automatic code generation

Automatic code generation is a process of deriving programs directly from a design representation. Recent initiatives such as Model Driven Architectures mean they are becoming an essential component of software engineering and many commercial tools now provide this capability. Whilst these tools provide greater flexibility and responsiveness in design, they are also largely unqualified with res...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

ژورنال

عنوان ژورنال: Proceedings of the ... AAAI Conference on Artificial Intelligence

سال: 2021

ISSN: ['2159-5399', '2374-3468']

DOI: https://doi.org/10.1609/aaai.v35i16.17662